IAPR: Final project - Chocolate Recognition¶
Moodle group ID: 3 Kaggle challenge: Deep learning Kaggle team name (exact): "Byte the Bar"
Author 1 (sciper): Nathann Morand (296190)
Author 2 (sciper): David Croce (327277)
Author 3 (sciper): Felipe Ramirez (331471)
Due date: 21.05.2025 (11:59 pm)
Key Submission Guidelines:¶
- Before submitting your notebook, rerun it from scratch! Go to:
Kernel>Restart & Run All - Only groups of three will be accepted, except in exceptional circumstances.
Introduction¶
We are tasked to make a program that is able to count how many instance among 13 praline class in a cluttered image. We must retrain our model from scratch and are provided with only a very limited number of training image (90) The score is computed using a modified F1 score (that take difference in number of predicted praline)
For our approach we chose to make convolutional model based of the yolo architecture but instead we rewrote the network head to directly predict the number of instance for each class. We named our architecture yoco : you only count once. To train it we chose to make a synthetic dataset generator based of cropped praline from the training dataset pasted on top of the empty background that where extracted.
Dataset & Preprocessing¶
The original dataset offer 90 image that are 6000x4000 px, .JPG The image where taken in similar lightning condition and are relatively well lit. The inference dataset has the same properties.
EDA¶
Image from the dataset look like the following with different background object, different miscellaneous object scatter around and a few praline.
Using the provided CSV we computed the histogram of number of chocolate per image and the histogram showing the number of instance per class to see how well the class are balanced. We also show how many individual instance of praline are available across the dataset and the maximum number of chocolate of each class present on an image.
import pandas as pd
import matplotlib.pyplot as plt
# Load the CSV file
df = pd.read_csv('src/dataset/dataset_project_iapr2025/train.csv')
# Calculate the total number of chocolates per image
df['total_chocolates'] = df.iloc[:, 1:].sum(axis=1)
# Print the total number of chocolates in the dataset
total_chocolates_in_dataset = df['total_chocolates'].sum()
print(f"Total number of chocolates in the dataset: {total_chocolates_in_dataset}")
# Get the maximum number of instances per class
max_per_class = df.iloc[:, 1:].max()
# Print the results
print("Maximum number of instances for each chocolate class in a single image:")
print(max_per_class)
# Plot the histogram for total chocolates per image
plt.figure(figsize=(12, 6))
plt.hist(df['total_chocolates'], bins=range(df['total_chocolates'].min(), df['total_chocolates'].max() + 1), edgecolor='black')
plt.title('Histogram of Total Chocolates per Image')
plt.xlabel('Total Number of Chocolates')
plt.ylabel('Frequency')
plt.grid(True)
plt.show()
# Plot the histogram for class distribution (excluding total chocolates column)
class_counts = df.iloc[:, 1:13].sum(axis=0)
plt.figure(figsize=(12, 6))
class_counts.plot(kind='bar', color='skyblue', edgecolor='black')
plt.title('Class Balance Histogram')
plt.xlabel('Chocolate Class')
plt.ylabel('Number of Chocolates')
plt.xticks(rotation=45)
plt.grid(True)
plt.show()
Total number of chocolates in the dataset: 584 Maximum number of instances for each chocolate class in a single image: Jelly White 3 Jelly Milk 3 Jelly Black 3 Amandina 2 Crème brulée 4 Triangolo 3 Tentation noir 4 Comtesse 5 Noblesse 3 Noir authentique 3 Passion au lait 3 Arabia 3 Stracciatella 3 total_chocolates 13 dtype: int64
Instance extraction¶
To make the synthetic dataset generator, we cropped manually the 583 praline present in the 90 image using a helper script to draw the box and save it in a new file. We made a second helper file to show the image and moving it to the correct folder after the operator write the class id thus making the sorting faster.
Once the praline where cropped we spent many hours cleaning the background from the 584 pralines using paint or Gimp. That being done we made another helper script to re-orient, center and rescale the praline. The recalling factor allowed use to measure the size variation between the praline and thus know that the variation was +-20% and thus a single detection head would be sufficient. We also did the same with the misc object present and patched the hole in the background.
Here are an overview of the cleaned praline :
import os
import matplotlib.pyplot as plt
from PIL import Image
# Define path and ignored folders
base_path = 'src/dataset/praline_clean'
ignored_folders = {"MiscObjects", "raw_praline", "references", "Background"}
# Get valid subfolders
valid_folders = [f for f in os.listdir(base_path) if os.path.isdir(os.path.join(base_path, f)) and f not in ignored_folders]
# Function to display a 4x4 image mosaic
def display_mosaic(images, title):
fig, axes = plt.subplots(4, 4, figsize=(8, 8))
fig.suptitle(title, fontsize=16)
for i in range(16):
ax = axes[i // 4, i % 4]
if i < len(images):
ax.imshow(images[i])
ax.axis('off')
plt.tight_layout(rect=[0, 0, 1, 0.96])
plt.show()
# Process each valid folder
for folder in valid_folders:
folder_path = os.path.join(base_path, folder)
image_files = [f for f in os.listdir(folder_path) if f.lower().endswith(('.png', '.jpg', '.jpeg'))]
image_files = image_files[:36] # Limit to first 36 images
images = []
for img_file in image_files:
img_path = os.path.join(folder_path, img_file)
try:
img = Image.open(img_path).convert('RGB')
img = img.resize((200, 200))
images.append(img)
except Exception as e:
print(f"Error loading image {img_file}: {e}")
display_mosaic(images, title=folder)